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ABSTRACT 


Within  the  past  50  years  containerization  and  globalization  have  driven  a  change 
from  small  container  tenninals  to  large  container  terminals  that  need  efficient  logistic 
models  to  keep  up  with  the  significant  growth  in  container  traffic.  Efficiently  managing 
of  rubber  tired  gantry  cranes  and  planning  container  placement  within  the  terminal  are 
two  ways  to  increase  the  overall  efficiency  of  a  terminal.  In  this  thesis,  we  combine  these 
strategies  in  a  real-time  dispatching  tool  using  an  approximate  dynamic  programming 
heuristic.  The  heuristic  re-op timizes  at  the  rate  the  quay  crane  handles  containers, 
incorporating  endogenous  and  exogenous  information  in  each  solution.  We  formulated 
and  solved  an  Integer  Linear  Program  (ILP)  to  estimate  the  heuristic’s  solution  quality. 
The  heuristic  finds  solutions  within  seconds  and  the  absolute  gap  between  the  heuristic 
solution  and  the  ILP  solutions  remained  essentially  constant  as  the  size  of  the  problem 
increased. 
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EXECUTIVE  SUMMARY 


Due  to  containerization  and  globalization,  the  shipping  industry  has  seen 
significant  growth.  The  growth  that  the  shipping  industry  is  experiencing  can  also  be 
seen  in  container  tenninals  through  an  increase  in  the  volume  of  containers  that  have  to 
be  handled  and  processed.  With  record  growth  and  limited  space,  container  tenninals 
need  efficient  models  of  their  operations. 

The  commonly  accepted  performance  measure  of  a  container  terminal  is  the  time 
a  container  ship  is  berthed.  The  operation  within  the  tenninal  that  can  most  impede  a 
timely-tum  around  of  a  container  ship  is  the  movement  of  the  rubber  tired  gantry  cranes 
(RTG).  Due  to  this  fact,  it  is  critical  to  plan  the  efficient  movement  of  RTGs  within  a 
container  terminal.  Also,  when  an  import  container  is  removed  from  a  container  ship 
there  are  a  multitude  of  places  within  the  tenninal  that  a  container  can  be  temporarily 
stored.  Terminals  have  internal  policies  called  grounding  policies  that  detennine  where 
to  place  a  container.  Like  planning  the  efficient  movement  of  RTGs,  container  terminal 
wants  to  establish  a  grounding  policy  that  will  aid  in  reducing  a  containers  ship’s  berth 
time.  Traditionally,  the  grounding  policy  is  found  independently  of  the  efficient 
movement  of  RTGs. 

In  this  thesis  we  develop  a  model  to  find  a  grounding  policy  in  conjunction  with 
the  efficient  movement  of  RTGs  to  increase  the  throughput  of  import  and  export 
containers  and  to  reduce  a  container  ship’s  berthing  time.  The  objective  is  achieved 
through  a  deterministic  real-time  dispatching  model  solved  using  an  approximate 
dynamic  programming  (ADP)  heuristic.  Both  the  grounding  policy  and  the  movement  of 
RTGs  are  optimized  simultaneously  in  order  to  find  an  approximately  optimal  solution. 
The  output  of  the  model  is  a  grounding  policy  and  recommended  RTG  movement  for  a 
given  planning  horizon.  The  model  is  reoptimized  in  real  time  using  a  rolling  planning 
horizon  and  updated  forecasts  of  required  container  movement  and  RTG  availability. 

We  formulate  an  integer  linear  program  (ILP)  to  bound  the  optimal  value  of  the 
RTG  movement  and  ground  policy  optimization  problem.  The  ILP  is  a  relaxation  of  our 


xi 


problem  and  provides  a  lower  bound.  Additionally,  there  is  a  prior  formulation  that 
models  a  restriction  of  the  same  problem  and  we  use  it  to  establish  upper  bounds  to  our 
problem. 

The  ADP  based  heuristic  solves  problem  instances  with  a  planning  horizon  of  up 
to  100  containers  in  less  than  30  seconds  for  six  RTGs  covering  36  blocks.  Based  on 
solve  times,  our  ADP  heuristic  is  suitable  for  real-time  dispatching  of  container  terminal 
assets.  The  objective  function  values  obtained  using  the  heuristic  are  well  below  the 
upper  bound  from  the  prior,  restrictive  model.  Additionally,  the  objective  values 
obtained  from  the  heuristic  follow  the  same  trends  as  the  lower  bounds  from  the  baseline 
model  as  the  planning  horizon  increases. 
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I.  INTRODUCTION 


A.  CONTAINERIZATION 

Containerization,  the  practice  of  loading  goods  into  standardized  sealable 
containers  for  overseas  shipment  on  large  container  ships,  was  developed  in  the  1960s 
and  makes  it  much  easier  to  move  bulk  goods  from  port  to  port  by  reducing  the  time  and 
labor  required  to  load  and  offload  container  ships.  Containerization  has  made  the 
shipping  industry  more  profitable  and  competitive  (Levinson,  2006),  and  has  enabled  the 
globalization  of  many  other  industries.  Due  to  these  two  factors,  the  shipping  industry 
has  changed  significantly  within  the  past  50  years  and  is  still  experiencing  significant 
growth.  In  the  United  States  alone,  container  traffic  has  tripled  in  volume  from  1995  to 
2006  and  has  grown  10%  alone  from  2005  to  2006.  It  is  also  estimated  that  90%  of  the 
volume  of  world  trade  is  handled  at  one  point  by  the  shipping  industry  (U.  S.  Department 
of  Transportation,  2007). 

A  container  is  a  large  metal  box  used  to  store  goods  for  shipment  overseas. 
Containers  come  in  a  small  number  of  standard  sizes  to  make  their  handling  easier.  A 
container  ship  is  a  merchant  vessel  that  is  engineered  to  optimally  transship  containers. 
A  container  terminal  (or  simply  terminal)  is  a  facility  in  a  port  that  manages  the  loading, 
unloading  and  temporary  storage  of  shipping  containers.  There  are  hundreds  of  container 
tenninals  located  at  large  ports  worldwide.  Shipping  containers  arrive  at  and  depart  from 
a  container  tenninal  via  container  ships,  trucks  and  trains.  For  a  container  terminal  to  be 
competitive,  it  must  keep  shipping  costs  down  by  performing  its  operations  efficiently. 

In  this  thesis,  we  focus  on  the  movement  of  containers  to  and  from  container 

ships.  When  an  import  container  arrives  on  a  container  ship,  the  container  will  be 

temporarily  stored  in  the  yard  of  a  container  terminal.  After  the  container  is  stored,  it  is 

exported  via  one  of  the  above  modes  of  transportation.  All  containers  stored  within  the 

yard  for  future  loading  onto  other  container  ships  are  export  containers.  The  yard  of  a 

container  terminal  is  made  up  of  series  of  container  blocks,  each  block  is  an  area  where 

many  containers  are  stacked  and  stored  together.  Each  block  has  a  buffer  area  which  is  a 
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queue  for  containers  waiting  to  be  placed  within  the  block.  A  series  of  adjacent  blocks 
makes  up  a  lane.  The  quayside  or  berth  is  the  area  of  the  terminal  that  a  container  ship  is 
berthed  and  quay  cranes  are  positioned  to  load  and  discharge  containers  from  a  container 
ship  (Figure  1). 


Quay  Crane 

Container  Ship  y 

Terminal  _ ^ 

Tractor 

tae“  Quayside  (Berth) 

Container  Yard 


Lane 


RTG 


Block 


Figure  1.  Container  Terminal  Layout:  A  schematic  overhead  view  of  a  container 
terminal  that  has  12  lanes  and  36  blocks.  The  berth  has  a  container  ship  that  has  a  quay 
crane  to  handle  the  import  and  export  container  sequence.  The  containers  are  moved 
between  the  berth  and  the  yard  on  the  terminal  tractors.  Within  the  yard  RTGs  are 
moving  export  containers  from  blocks  to  terminal  tractors,  and  vice-versa  for  import 
containers. 


1.  Terminal  Equipment 

The  equipment  within  a  container  terminal  varies  from  facility  to  facility.  In 
order  to  improve  the  efficiency  of  container  facilities,  we  investigate  the  operations  of 
quay  cranes,  rubber  tired  gantry  cranes  and  tenninal  tractors. 

The  quay  crane  (Figure  2)  is  an  essential  part  of  operations  at  a  terminal.  The 
quay  crane  is  located  in  the  berth  of  a  terminal  and  is  used  to  load  and  discharge 
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containers  from  a  container  ship.  In  the  load  operations,  export  containers  are  moved 
from  the  yard  to  the  quay  crane  via  a  terminal  tractor.  The  quay  then  removes  the  export 
container  from  the  terminal  tractor  to  place  it  onboard  a  container  ship.  For  the  discharge 
operation,  an  import  container  is  removed  from  a  container  ship  by  a  quay  crane  and 
placed  on  a  terminal  tractor  for  transit.  The  terminal  tractor  moves  the  container  to  an 
appropriate  location  within  the  yard  for  temporary  storage.  The  quay  crane  can  vary  the 
rate  at  which  it  loads  and  discharges  containers.  This  variable  rate,  called  the  push  rate, 
can  be  slowed  to  help  prevent  or  alleviate  congestion  within  the  container  terminal. 


Figure  2.  Quay  crane:  Quay  cranes  loading  and  discharging  containers  from  a 
container  ship  (From  Container  Shipping  Infonnation  Service,  2008). 

Within  the  yard  of  the  terminal  the  assets  perfonning  the  majority  of  the  work  are 
the  rubber  tired  gantry  cranes  (RTG)  and  the  terminal  tractors  (Figure  3).  A  tenninal 
tractor  moves  a  single  container  between  a  quay  crane  and  a  RTG.  A  RTG  is  a  crane 
that  moves  from  block  to  block  working  on  containers.  The  work  of  a  RTG  is  defined  as 
moving  import  containers  off  terminal  tractors  to  place  them  within  a  block  and  vice 
versa  for  export  containers. 
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Figure  3.  Rubber  Tired  Gantry  Crane  and  Terminal  Tractor:  A  terminal  tractor 
in  the  buffer  area  waiting  on  a  RTG.  The  RTG  will  move  the  containers  from  the 
terminal  tractor  to  place  the  container  in  the  block  (From  Container  Shipping  Information 
Service,  2008). 

2.  Operations 

Within  the  terminal,  minimizing  the  berth  time  of  a  container  ship  is  not  as  simple 
as  buying  more  cranes.  Buying  more  quay  cranes  for  a  terminal  so  that  more  work  can  be 
accomplished  is  cost  prohibitive,  and  placing  more  RTG  cranes  in  the  yard  could 
potentially  slow  a  terminal’s  operations  due  to  traffic  congestion.  In  the  daily  operations 
of  a  container  terminal,  RTGs  are  frequently  the  cause  of  delays.  The  efficient  operation 
of  each  RTG  is  the  key  to  expediting  the  movement  of  import  and  export  containers  and 
reducing  the  berthing  time  of  a  container  ship. 

The  focus  in  this  thesis  is  placed  on  the  RTG  operation  because  of  the  potential  it 

has  to  slow  terminal  operations  if  their  operation  is  not  planned  appropriately.  A  RTG  is 

not  constrained  to  one  work  block,  but  is  free  to  move  to  other  work  blocks  as  necessary. 

The  movement  of  RTGs  is  essential  in  the  task  of  completing  the  work  on  the  import  and 

export  containers,  but  it  comes  at  a  price.  The  movement  is  a  timely  operation  that 
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obligates  additional  personnel,  causes  congestion  within  the  terminal,  and  reduces  the 
effectiveness  of  a  RTG  because  it  is  not  handling  containers  during  the  move.  Figures  4 
and  5  are  two  examples  of  RTG  transitions.  The  more  complex  the  move  that  a  RTG 
makes,  the  greater  the  potential  there  will  be  a  reduction  in  tenninal  efficiency. 

Container  tenninals  have  different  policies  and  procedures  for  detennining  where 
an  import  container  is  to  be  temporarily  store  in  the  yard.  The  rule  for  determining  this 
container  placement  is  referred  to  as  the  grounding  policy.  The  grounding  policy 
typically  groups  containers  with  similar  characteristics  such  as  content,  origin,  and 
destination.  A  grounding  policy  that  is  enacted  in  conjunction  with  the  dispatching  or 
movement  of  the  RTGs  will  result  in  a  more  efficient  planning  process. 

RTG 

Initial  Location 


RTG 

Final  Location 


Figure  4.  Complex  RTG  Move:  A  RTG  transitioning  to  a  block  in  an  adjacent  lane 
requires  a  RTG  to  turn  90°  on  its  wheels  twice. 

RTG 

Initial  Location 


Figure  5.  Simple  RTG  Move:  A  RTG  transitions  to  an  adjacent  block  within  its 
lane.  The  move  does  not  require  a  RTG  to  turn  90°  on  its  wheels  and  is  the  most  efficient 
move  between  blocks. 


RTG 

Final  Location 
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B. 


RESEARCH  GOAL  AND  APPROACH 


In  order  to  improve  the  efficiency  of  RTGs  in  a  terminal,  we  have  developed  a 
mathematical  model  of  the  tenninal  operations  that  directly  involve  container  ship  cargo. 
We  present  this  model  and  a  real-time  dispatching  tool  that  uses  approximate  dynamic 
programming  (ADP)  to  find  efficient  solutions.  Our  model  helps  improve  the  operations 
of  the  RTGs  in  a  terminal,  thereby  increasing  the  throughput  of  import  and  export 
containers  and  reducing  each  container  ship’s  berthing  time.  The  real-time  dispatching 
tool  will  take  as  input  a  container  sequence  (i.e.,  a  known  sequence  of  “jobs”  for  the  quay 
crane)  and  will  provide  as  output  a  grounding  policy  for  all  import  containers  and 
recommended  RTG  moves.  The  tool  will  ensure  that  all  export  containers  are  moved  out 
of  the  blocks  to  terminal  tractors  that  will  take  them  to  the  quay  crane  for  loading 
onboard  a  container  ship.  The  tool  will  also  ensure  that  all  import  containers  are  placed 
into  the  blocks  by  a  RTG.  The  objective  of  the  real-time  dispatching  tool  is  to  complete 
the  work  associated  with  the  given  container  sequence  in  minimum  time. 

The  decision  points  in  the  dynamic  program  (DP)  are  regularly  spaced  in  time, 
with  the  spacing  detennined  by  the  quay  crane  push  rate,  which  is  fixed  at  60  seconds  in 
our  examples.  At  each  decision  point,  the  tool  will  determine  a  grounding  policy 
(destination  of  containers  on  terminal  tractors)  for  the  next  container  that  is  being 
discharged  from  a  container  ship.  In  addition  to  the  grounding  policy  the  tool  will  also 
detennine  the  movement  and  handling  sequence  of  the  RTGs  for  import  and  export 
containers  within  a  container  terminal.  The  DP  is  detenninistic;  all  input  data  is  assumed 
to  be  known.  However,  to  handle  changes  that  occur  in  real-time  the  tool  will  be  used 
with  re-optimization  at  each  decision  point  incorporating  any  new  data  at  that  time. 
Some  examples  of  the  exogenous  processes  that  could  have  an  impact  on  operations  are 
changes  to  a  container  sequence,  delays  in  the  transit  of  RTGs  or  terminal  tractors  due  to 
congestion,  or  mechanical  failures  that  remove  assets  from  operation. 

We  solve  the  DP  using  a  heuristic  algorithm  based  on  approximate  dynamic 
programming  (ADP).  At  every  decision  point,  the  DP  will  provide  solutions  for  a  given 
planning  horizon,  which  is  the  number  of  containers  in  the  container  sequence  the  DP 
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will  solve  for.  The  resulting  solution  provides  a  rule,  or  policy,  for  efficient  movement  of 
containers  via  terminal  tractors  (a  grounding  policy)  in  conjunction  with  the  efficient 
movement  of  RTGs.  Overall  the  tool  will  result  in  reduced  berthing  time  for  container 
ship,  reduced  traffic  congestion  in  a  container  terminal  and  reduced  work  preformed 
within  a  container  tenninal. 

Since  the  DP  is  solved  using  a  heuristic,  a  comparison  is  needed  to  verify  solution 
quality.  We  formulate  an  integer  linear  program  (ILP)  for  baseline  comparison. 
Additionally,  the  optimal  values  and  computational  times  for  ADP  are  compared  with 
results  from  a  previous  ILP  developed  by  Akel  (2007).  The  two  integer  linear  programs 
will  give  a  measurement  of  perfonnance  for  the  ADP  algorithm. 

C.  STRUCTURE  OF  THESIS  AND  CHAPTER  OUTLINE 

This  thesis  is  divided  into  five  chapters,  including  the  Introduction.  Chapter  II 
gives  information  on  developing  events  and  studies  in  the  area  of  container  terminal 
optimization.  The  under  lying  decision  problem  is  discussed  and  developed  in  Chapter 
III  and  then  the  DP  and  ILP  are  presented.  In  Chapter  IV,  we  present  a  scenario  with  a 
discussion  of  the  data  set  which  was  provided  for  the  use  in  numerical  tests.  Following 
the  scenario  development  is  a  comparative  analysis  of  the  numerical  results.  Chapter  V 
gives  conclusions  based  on  the  findings  presented  in  this  thesis  in  and  recommendations 
for  future  work. 
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II.  DEVELOPMENTS  IN  CONTAINER  TERMINAL 

OPTIMIZATION 

A.  INTRODUCTION 

There  is  a  large  degree  of  coordination  and  planning  to  make  operations  run 
efficiently  at  container  tenninals.  Optimizing  operations  within  the  container  terminal 
yard  is  considered  one  of  the  most  complex  problems  in  the  industry  (Vis  et  ah,  2003).  In 
this  chapter,  we  review  some  analytic  approaches  to  container  tenninal  optimization. 

B.  CURRENT  DEVELOPMENTS 

Kim  et  al.  (2003)  examines  the  problem  of  scheduling  quay  cranes  within  a 
container  terminal.  They  present  a  heuristic  to  identify  a  load  and  discharge  sequence 
for  a  quay  crane.  When  identifying  the  sequence  of  containers  to  load  and  discharge,  the 
objective  is  to  minimize  the  container  ship  berth  time.  The  optimal  solution  of  the  load 
and  discharge  sequence  is  found  in  isolation  of  other  events  within  a  container  terminal. 

In  the  area  of  yard  resource  planning  there  are  two  topics  of  interest:  (i) 
identifying  the  destination  locations  of  the  import  containers  or  the  grounding  policy  and 
(ii)  the  scheduling  of  the  yard  assets  such  as  RTGs  (Taleb-Ibrahimi  et  al.,  1993).  Taleb- 
Ibrahimi  et  al.  (1993)  discusses  different  container  storage  strategies  and  the  handling 
requirement  that  is  coupled  with  the  strategies. 

Zhang  et  al.  (2002)  explores  RTG  deployment  based  on  workload  requirements. 
Their  approach  formulates  the  problem  as  a  mixed  integer  program;  however,  the 
problem  is  solved  heuristically  by  Lagrangean  relaxation  in  order  to  improve  solution 
times.  The  study  was  limited  to  20  blocks  and  solutions  were  found  for  four  hour 
planning  horizons,  with  the  remaining  unhandled  work  to  the  next  time  window.  Linn 
and  Zang  (2003)  developed  a  similar  RTG  deployment  heuristic  based  on  workload 
forecast.  The  scope  of  the  problem  was  limited  to  10  blocks  and  10  RTGs  to  keep  the 
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solve  time  of  the  mixed  integer  program  reasonable.  Their  heuristic  produces  near-optimal 
solutions  in  seconds.  In  both  of  the  above  studies,  the  number  of  RTGs  and  blocks  are  kept 
equal. 

Murty  et  al.  (2005)  discusses  different  decision  support  tools  to  be  used  in  all  aspects 
of  container  terminal  operations.  One  performance  measure  that  is  of  particular  interest  is  a 
qualitative  measure  of  congestion.  The  strategy  is  to  monitor  the  flow  of  the  traffic  on  the 
routes  traveled.  When  the  flow  reaches  an  upper  boundary,  this  indicates  an  increase  in 
potential  traffic  congestion. 

Ng  (2005)  developed  a  dynamic  programming  heuristic  to  schedule  multiple  yard 
cranes  within  a  single  zone.  A  zone  is  defined  as  the  set  of  blocks  that  a  RTG  is  constrained 
to.  The  focus  was  to  find  the  optimal  movement  of  yard  cranes  while  avoiding  the 
interference  of  other  cranes  to  reduce  terminal  tractor  waiting  time.  The  scope  of  the  study 
was  limited  to  two  yard  cranes  and  two  blocks. 

Linn  et  al.  (2003)  developed  a  RTG  deployment  algorithm  for  an  integer  linear 
program  to  find  the  optimal  deployment  of  RTG.  A  unique  feature  in  the  program  allowed 
RTGs  to  temporarily  store  a  container  and  then  return  to  the  container  for  optimal  placement 
within  the  yard.  The  scenarios  implemented  had  at  least  68  blocks  and  68  RTGs.  The 
algorithm  found  solutions  for  four-hour  windows  cascading  the  remainder  work  to  the  re¬ 
optimization  for  the  next  four-hour  time  window. 

C.  FUTURE  OF  CONTAINER  TERMINAL  OPTIMIZATION 

There  have  been  significant  developments  in  optimization  of  container  logistics. 
Some  of  the  developments  range  from  optimizing  the  quay  crane,  managing  the  stowage  plan 
for  a  container  ship  and  optimizing  the  movement  of  RTGs  in  a  terminal.  There  has  been  no 
specific  focus  on  interrelating  the  different  yard  operations  and  optimizing  multiple  yard 
operations  simultaneously.  For  example,  it  may  be  optimal  for  a  quay  crane  to  discharge  a  set 
of  containers  before  handling  the  subsequent  set.  However,  if  the  destination  of  all 
containers  in  the  first  set  is  the  same,  then  terminal  congestion  may  be  unavoidable. 
Integrated  operations  need  to  be  considered  in  the  high  performance  environment  of 
container  terminals  (Steenken  et  al.,  2004).  This  thesis  is  a  first  attempt  to  optimize 
simultaneously  both  the  grounding  policy  and  the  RTG  movement  policy. 
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III.  MODEL  DEVELOPMENT 


A.  MODEL  DEVELOPMENT 

There  are  many  interrelated  aspects  of  a  container  tenninal  that  can  have  an 
impact  on  performance  ( i.e terminal  layout,  types  of  equipment  used,  external  truck 
operations,  and  tenninal  tractor  operations).  All  operations  within  a  terminal  are 
important  and  interrelated,  but  the  computational  complexity  of  a  model  that  considers 
the  multiple  objectives  of  the  different  operations  prevents  us  from  proceeding  in  this 
direction.  Since  RTGs  are  considered  the  bottleneck  of  container  terminal  operations,  we 
choose  to  limit  the  scope  of  the  problem  by  looking  specifically  at  a  grounding  policy  in 
conjunction  with  the  movement  of  RTGs.  The  model  will  increase  the  throughput  of 
import  and  export  containers  and  reduce  a  container  ship’s  berthing  time. 

In  order  to  sequence  the  movement  of  the  RTGs  with  the  model,  a  container 
sequence  is  put  together  by  the  container  terminal  personnel.  Before  a  container  ship 
pulls  into  a  port,  terminal  personnel  know  the  contents,  destination  and  position  in  the 
discharge  sequence  of  each  import  container.  Based  on  the  terminals  internal  rules  on 
grounding  a  container  within  a  tenninal,  personnel  build  a  list  of  different  areas  where  a 
container  can  be  stored  in  the  terminal.  Additionally,  the  personnel  know  the  location  of 
the  containers  that  are  to  be  exported  onto  an  incoming  container  ship.  With  the  above 
information  they  build  a  sequenced  list  of  containers  to  be  discharged  and  loaded  onto 
this  ship  by  the  quay  crane 

The  data  for  the  model  is  put  together  before  a  container  ship  arrives.  With  the 
sequenced  list  of  containers  and  list  of  potential  grounding  areas  for  each  import 
container,  our  objective  is  to  find  the  most  efficient  sequencing  of  RTG  work  and 
grounding  policy  to  satisfy  the  constraint  of  the  container  sequence.  However,  in  the 
development  of  a  model  to  meet  this  objective,  we  made  some  assumptions  in  order  to 
simplify  the  problem. 
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For  the  terminal  tractors,  we  specifically  choose  not  to  model  their  movement  but 
instead  assume  that  they  are  a  freely  available  resource.  With  this  choice,  we  do  not  have 
to  consider  the  scheduling  of  terminal  tractors  and  we  can  assume  that  the  time  that  it 
takes  a  tenninal  tractor  to  traverse  a  route  is  deterministic,  and  based  on  the  distance 
traveled.  We  acknowledge  that  a  terminal  tractor  can  be  delayed  due  to  congestion 
within  the  yard  or  due  to  exogenous  processes,  however,  the  time  of  delay  is  negligible 
compared  to  the  time  it  takes  to  position  a  RTG  within  the  yard  to  handle  a  container  on  a 
tenninal  tractor. 

Within  the  model,  RTGs  are  allowed  to  transit  to  any  block  within  the  yard.  The 
RTGs  are  not  limited  to  move  to  specific  set  of  blocks.  However,  RTGs  are  not  allowed 
to  occupy  the  same  block  at  the  same  time. 

Each  block  in  a  container  terminal  has  an  associated  buffer,  a  first-in-first-out 
queue  that  contains  import  containers  waiting  for  a  RTG  to  place  them  in  the  block.  The 
buffer  also  contains  empty  tenninal  tractors  that  are  waiting  for  an  export  container  to 
transit  to  the  quay  cane. 

Our  final  major  assumption  is  made  on  the  quay  crane  push  rate.  The  push  rate  is 
the  rate  that  a  quay  crane  is  moving  containers  to  and  from  a  container  ship.  The  quay 
crane  push  rate  can  be  varied  in  order  to  alleviate  congestion  within  the  yard,  however, 
we  assume  the  rate  is  constant  at  one  unit  of  time  (one  minute). 

B.  DYNAMIC  PROGRAM  FORMULATION 

We  first  formulate  the  RTG  Operations  Problem  (RTGOP)  as  a  dynamic  program 
(DP).  The  DP  seeks  to  minimize  the  total  amount  of  time  until  a  given  sequence  of 
containers  have  been  moved  to  their  respective  destinations  within  the  terminal  or 
onboard  a  ship.  The  work  is  composed  of  import  and  export  containers.  Work  on  an 
import  container  consists  of  moving  a  container  from  a  container  ship  to  a  block  and 
work  on  an  export  container  consists  of  moving  a  container  from  a  block  to  a  container 
ship. 
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The  formulation  of  the  DP  is  as  follows: 


Indices/Sets 

t  eT  =  {1,2,.. .,00} 

s 

P 

w,w'eW  =  {  1,2,...,\W\} 

b, b'eB  =  {l,2,...,\B\} 

1,1'  e  L  =  W  u  B  u  {p}  u  {5} 
g,g'zG  =  {l,2,...,\G\} 
a  e  A  =  { 0,1} 
r  £  R  =  W  x  [0,  go) 
q  £  Q  =  L  x  [0,  go)  x  A 
W(b )  <z  W 
B(w)  c  B 

ci,Ci'£Ci  =  {\,2,...,\Ci\) 
ce,ce'£Ce  =  {\,2,..\Ce\) 

c, c'£C  =  CivjCe 

Data 

jHAND 

.RIG 

lw,w' 

jCON 
t  AVAIL 


Discrete  time  step  which  represents  the  decision  points  in  a 
continuous  time  process. 

Container  location  representing  onboard  a  container  ship. 
Container  location  representing  at  a  quay  crane. 

Blocks  that  RTGs  can  occupy. 

Buffer  areas  for  containers. 

Locations  for  containers. 

RTGs. 

Type  of  shipping  container  (0  =  import  and  1  =  export). 
RTG  attribute  vectors. 

Container  attribute  vectors. 

Block  w  that  corresponds  with  buffer  b. 

Buffer  b  that  corresponds  with  block  w. 

Import  containers. 

Export  containers. 

Container  sequence  to  be  imported  or  exported. 


Time  for  a  RTG  to  move  a  container  from  a  terminal  tractor 
to  a  block  of  containers  or  vice-versa. 

Time  to  move  a  RTG  from  block  w  to  block  w\ 

Time  to  move  a  container  between  locations  /  and  /'. 

Time  of  the  day  when  import  container  cI  is  scheduled  to 
be  available  at  the  quay  crane  for  transport  to  the  yard. 
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Time  of  the  day  when  export  container  cE  is  scheduled  to 

be  loaded  onto  the  container  ship  by  the  quay  crane. 

Length  of  time  interval  t  (in  our  example  A(t)=l  minute 
for  all  t) 

The  following  variables,  states,  and  functions  are  all  defined  at  any  time  t  in  the  planning 
horizon. 

Decision  Variables 

Xt  g  e  W  Selected  block  for  RTG  g. 

Ht  c  e  {0,1}  1  if  RTG  g  assigned  to  start  work  on  container  c. 

Yt  c  eL  Selected  location  for  container  c. 

[K-L  ' 

Z ,  =  ( H,  „  „ )  Vector  of  decisions. 

*  V  >’g’C  JgcG,CGC 

v  (YX.c  , 

RTG  attribute  vector  where  w  is  the  current  block  or 
destination  block  of  RTG  g  and  z  is  the  time  until  RTG  g 
becomes  available,  r  >  0  indicates  that  the  RTG  is 
unavailable  and  r  =  0  indicates  that  the  RTG  is  available 

Container  attribute  vector  where  /  is  the  current  location  or 
destination  location  of  container  c  and  cr  is  the  time  until 
container  c  becomes  available,  a  >  0  indicates  the 
container  is  unavailable  and  cr  =  0  indicates  the  container 
is  available.  The  container  type  is  indicated  by  a,  where  1 
is  an  export  and  0  is  an  import. 


States 
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State  vector 


S,= 


((r  )  ^ 

V'xJgeG 

Me*  V 


Constraints 

Nr(s,)  if  Yj Ht,g,c  =  0  and  T,,g  =  0 

C 

i wt,g>  otherwise 


Vg 


Where  NR(St )  aW  is  a  neighborhood  function  for  a  RTG;  it  is  the  set  of  blocks  an  RTG 
can  be  sent  to  given  state  vector  St. 

W  ift<t^L 

Nc (St )  if  t  >  t*fAU' ,  fCi ,  e  {4 ,  fCi  £  {s}  and  1ICr  e  {5}  Vc7  and  cE<cf 

ifE*w=1 

ife,'}  otherwise 


0)^.' 


Vc7 ' 


Where  Nc(Sr)  a  L  is  a  neighborhood  function  for  a  container;  it  is  the  set  of  blocks  a 
container  can  be  sent  to  given  state  vector  St. 


{4 

if  Kce'  E  M’  ar,cE'=0'  kc,  £  W  and  W  G  ^  Vc^  and  C£  <  C£  ’ 

{p} 

if  f  ,  e  B  and  a.  ,  ,  =0 

g< 

t,CE  t,CE 

if2X«,v  =  > 

g 

ikJ 

otherwise 

c 

Vg 

VI 

too 

to'xl  60 

Vc 

(7)  ^ 

=  0 

if  Wf>g  *  If  (/7>C/ )  or  at  Ci  *  0  or  r,  g  *  0 

Vg,c7 

(8)^=0 

if  wtg  *  kcE  or  at,cE  *  0  or  f,g  *  0 

Vg,G 
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Transitions 

(9)»W=^ 

Vg 

o 

— 1 

+ 

r> 

II 

j-c 

n 

Vc 

(11)  max  {r,  s+X  +O„-A(0,  0) 

Vg 

(12)  =max{°t,c+TsH<,c,gtHAND+G,c  -mo,  0} 

Vc 

g 


Contribution  Function 


(13)  F(St,Zt) 


|0  if  Yti.  e  W  Vc7,  YUCf:  e  {.s'}  Vc£  and<r(+1>c=0  Vc 
[A(0  otherwise 


Value  Function 

(14)  V,(S,)  =  min(F(S„Z,)  +  VM(Sn,j) 


The  cost  function  (13)  describes  the  amount  of  additional  time  added  in  period  t 
based  on  the  current  state.  Note  that  when  all  work  is  complete  F(Sl,Zl)  =  0 .  The  value 

function  (14)  seeks  to  minimize  the  total  amount  of  time  that  is  spent  working  on  the 
containers.  Constraint  set  (1)  ensures  that  all  RTGs  g,g'  are  assigned  to  different  blocks. 
Constraint  set  (2)  allows  a  RTG  g  to  only  move  to  blocks  within  its  neighborhood  if  it  is 
not  scheduled  to  handle  a  container  and  if  it  is  available  at  time  t.  Otherwise  RTG  g  will 
remain  in  the  same  block.  Constraint  set  (3)  restricts  the  movement  of  import  container 
Cj .  The  quay  crane  does  not  move  container  c  from  a  container  ship  until  the  container  is 

available  in  the  handling  sequence  tf  AIL .  When  container  c  is  the  next  container  in 

sequence,  the  quay  crane  moves  the  container  from  the  container  ship  to  a  terminal 
tractor  and  the  terminal  tractor  then  transports  the  container  to  a  buffer  zone  to  await  a 
RTG.  RTG  g  moves  a  container  c  from  the  buffer  area  into  the  corresponding  work 
block  when  indicated  by  the  decision  HUgc.  Otherwise,  the  container  c  remains  in  the 
same  location  until  the  next  decision  point.  Constraint  set  (4)  restricts  the  movement  of  a 
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export  container  cE .  RTG  g  moves  a  container  c  from  a  block  w  to  a  terminal  tractor  in 

buffer  area  b  when  indicated  by  decision  HtygyC.  A  terminal  tractor  located  in  the  buffer 
area  b  moves  container  c  to  the  quay  crane  p  when  a  container  is  in  position  to  be  moved. 
The  quay  crane  moves  container  c,  when  the  container  is  next  in  sequence  tfAIL  and  the 

container  is  in  position  to  be  moved.  Otherwise,  container  c  remains  in  the  same  location 
until  the  next  decision  point.  Constraint  set  (5)  allows  at  most  one  container  c  to  be 
assigned  to  be  moved  by  a  RTG  g  at  time  t.  Constraint  set  (6)  allows  at  most  one  RTG  g 
to  be  assigned  to  move  a  container  c  at  time  t.  Constraint  set  (7)  does  not  allow  a  RTG  g 
to  handle  an  import  container  c  if  it  is  not  collocated  with  the  RTG  or  if  either  is 
unavailable  at  time  t.  Constraint  set  (8)  does  not  allow  a  RTG  g  to  handle  an  export 
container  c  if  they  are  not  collocated  or  if  either  is  unavailable  at  time  t.  Transition  set 
(9)  moves  RTG  g  to  a  block  Xtyg  at  time  t.  Transition  set  (10)  moves  a  container  c  to 
location  YtyC  at  time  t.  Transition  sets  (11)  and  (12)  assign  work  time  to  a  RTG  g  and  a 
container  c  respectively.  The  transition  sets  (11)  and  (12)  are  decremented  by 
A (t)  during  the  transition  from  t  to  the  next  time  period  t+ 1 .  The  max  function  in  these 

sets  does  not  allow  the  respective  Tt+1  or  cr,+1  to  fall  below  zero. 

C.  POLICY  DEVELOPMENT 

In  RTGOP,  solving  Bellman’s  equation  (14)  with  backward  recursion  is 
intractable  since  there  are  an  infinite  number  of  states  and  (theoretically,  although  not 
practically)  an  infinite  time  horizon.  So,  we  develop  a  heuristic  policy  founded  on  rule- 
based  policies. 

A  rule-based  policy  is  derived  from  common  operational  practices.  For  example, 
“move  an  available  RTG  to  the  nearest  container  that  needs  to  be  handled”,  or  “move  a 
container  to  where  a  RTG  is  positioned,”  are  both  examples  of  common  practices  that  can 
be  implemented  as  a  rule-based  policy  for  handling  containers  and  RTGs.  The  rules  that 
are  established  detennine  a  decision  based  on  the  endogenous  information  that  is 
available  at  that  time  period. 


17 


1.  Neighborhood  Function  Policies 


Our  policies  are  intended  to  restrict  the  set  of  destination  areas  and  container 
choices  in  (3)  and  (4)  in  our  DP  formulation.  The  set-functions  NR(St )  and  Nc(St )  are 

each  called  a  neighborhood  function,  because  they  indicate  a  set  of  feasible,  typically 
“nearby,”  choices  that  can  be  made  for  the  next  steps  involving  an  RTG  or  a  container, 
respectively.  We  try  to  develop  policies  to  constrain  the  neighborhood  functions  that 
minimize  the  time  to  complete  all  work;  our  policies  reflect  this  by  trying  to  minimize  the 
travel  time  of  each  RTG  (Figure  6).  We  made  this  choice  to  focus  on  RTG  travel  time 
rather  than  container  travel  time  because  the  travel  time  of  a  container  is  directly 
dependent  on  how  efficiently  a  RTG  transits  to  handle  that  container  (Figure  7). 


MINIMIZE 

RTG  Idle  Time 
(Waiting  to  Transit) 

RTG  Transition 
(Move  to  Next  Block) 

RTG  Idle  Time 
(Waiting  to  Handle) 

RTG  Handle  Time 

RTGs  1stwork 

RTGs  2nd  work 

RTGs  3rd  work 

assignment 

assignment 

assignment 

RTG  last  work 
assignment 


Figure  6.  RTG  Timeline:  The  sequence  of  jobs  or  block  assignments  for  a  RTG  and 
the  timeline  of  the  assignments. 
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Loaded  onto  Ship 

Block  by  RTG 

Berth 

for  Quay 

by  Quay  Crane 

Export  Container  Timeline 


1st  Container 
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(Export) 
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■  ■  ■ 

Last  Container 

Import  Container  Timeline 


Wait 

Discharged  from  Ship 

for  Quay 

by  Quay  Crane 

Transit  to 
Buffer  area 


Placed  in 
Block  by  RTG 


Figure  7.  Container  Timeline:  The  container  sequence  and  the  timeline  of  an 
export  container  and  a  timeline  of  an  import  container.  The  event  in  red  is  the  containers 
dependency  on  a  RTG. 
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a. 


RTGOP  Heuristic  Algorithm 


The  policies  that  constrain  the  blocks  a  RTG  can  move  to  are  defined  in 
the  neighborhood  function,  N R (S/)  a  W  .  We  define  NR(St )  as  the  blocks  that  have 

containers  that  are  waiting  in  the  buffer  to  be  handled  and  do  not  have  a  RTG  assigned  to 
them.  We  then  define  feasible  policies  as  those  assignments  of  available  RTGs  to  blocks 
in  NR(St )  that  do  not  assign  multiple  RTGs  to  the  same  block  during  the  same  time 

period.  We  nominate  three  separate  measures  of  perfonnance  to  evaluate  policies  at  each 
decision  epoch  t  to  enable  a  sequential,  myopic  heuristic  for  solving  RTGOP,  and  then 
we  define  a  modified  heuristic  with  limited  look-ahead  that  can  hold  work  for  a  worthy 
unavailable  RTG. 

Our  heuristic  algorithm  sequentially  solves  RTGOP  by  choosing  a 
destination  for  each  available  RTG,  one  time  period  at  a  time.  The  heuristic  starts  at  the 
first  time  period  t=  1,  and,  in  each  time  period,  it  examines  each  RTG  in  a  fixed  sequence. 
For  each  available  RTG  g,  the  algorithm  considers  each  block  beRG(St)  as  a  potential 

destination,  calculates  the  travel  time  for  g  to  each  b  and  it  saves  the  six  closest  blocks  for 
each  RTG.  It  then  enumerates  all  possible  selections  of  one  block  (out  of  the  six  saved 
blocks)  per  available  RTG.  If  all  six  RTGs  are  available,  this  leads  to  46,656  policies. 
The  algorithm  considers  each  of  these  policies  and  chooses  the  feasible  policy  (i.e.,  one 
that  does  not  have  multiple  RTGs  in  the  same  block  in  the  next  time  period)  that 
minimizes  our  selected  measure  of  perfonnance.  This  policy  is  implemented  and  fixed 
for  the  current  period,  and  then  the  next  decision  epoch  t  is  considered. 

We  introduce  three  separate  measures  of  performance  that  guide  our 
choice  of  a  single  policy  based  on  the  state  St: 

(1)  Minimize  the  Sum  of  RTG  Travel  Time.  For  each  feasible 
policy,  the  RTG  travel  times  are  summed,  and  we  choose  a  feasible  policy  that  results  in 
the  minimum  sum  of  travel  times. 

(2)  Minimize  the  Maximum  RTG  Travel  Time.  We  choose  a 
feasible  policy  that  minimizes  the  maximum,  over  all  RTGs,  of  their  travel  time. 
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(3)  Minimize  the  Maximum  RTG  Travel  Time  Plus  a  Reward. 
This  measure  of  performance  has  the  same  definition  outlined  in  Paragraph  (2)  with  the 
addition  of  a  time  reward  that  is  subtracted  from  a  RTG’s  travel  time  if  its  destination 
block  has  an  export  container.  The  export  container  that  determines  the  reward  is  the 
export  container  that  closest  to  becoming  available  tf  "AIL  in  the  container  sequence  C. 
The  reward  is  scaled  according  to  the  difference  in  the  of  the  terminal’s  current  time  t  and 
the  time  export  container  will  become  available  tf  'AIL .  The  reward  makes  blocks  that 
have  an  export  container  that  is  approaching  its  availability  time  look  more  appealing. 

b.  Container  Neighborhood  Function 

The  container  neighborhood  function  Nc  (St )  c  L  finds  a  grounding 

policy  if  the  next  container  to  become  available  t^VAIL  is  an  import  container.  All  of  the 

potential  grounding  locations  of  an  import  container  are  compared  with  each  of  the  RTGs 
to  find  a  grounding  location  that  adds  minimal  travel  time.  The  grounding  location  is 
compared  to  RTG’s  that  are  in  two  different  states;  a  RTG  that  is  preparing  to  move  to  a 
new  block  and  a  RTG  that  has  a  block  assignment. 

A  RTG  that  is  preparing  to  move  has  a  destination  block  and  a  travel  time 
assigned  by  the  RTG  neighborhood  function.  The  grounding  location  that  adds  the  least 
amount  of  travel  time  to  a  RTGs  current  travel  time  is  the  grounding  location  used  for 
comparison  to  all  other  RTGs.  The  change  in  travel  time  is  determined  from  comparing 
the  original  travel  time  to  the  RTG  traveling  to  the  grounding  location  before  and  after 
traveling  to  its  destination  block. 

The  change  in  travel  time  for  a  RTG  that  has  a  block  assignment  is  simply 
the  amount  of  time  it  takes  the  RTG  to  travel  to  the  grounding  location  from  its  current 
block.  The  grounding  location  that  has  the  smallest  change  in  travel  is  the  used  for 
comparison  to  all  other  RTGs. 

Across  all  the  RTGs,  the  grounding  location  that  has  the  smallest  change 
in  travel  time  to  a  RTG  detennines  the  grounding  policy.  If  a  grounding  policy  changes 
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the  sequence  of  travel  for  a  RTG  preparing  to  travel,  then  the  RTG’s  destination  block  is 
changed  to  the  grounding  policy  block. 

2.  Look-Ahead  Policy 

In  constraint  set  (2)  of  the  DP,  the  neighborhood  function  does  not  assign  a 
destination  block  to  a  RTG  that  is  unavailable.  However,  we  define  a  look-ahead  policy 
that  identifies  how  desirable  it  is  to  reserve  a  destination  block  for  an  unavailable  RTG,  g. 
When  calculating  g ’s  travel  time  to  a  destination  block,  the  travel  time  is  penalized  the 
amount  of  time  until  it  becomes  available,  rt  .  How  far  the  policy  looks  into  the  future 

is  the  availability  window.  If  the  availability  window  is  greater  than  a  RTG  unavailable 
time  zt  ,  then  g  is  allowed  to  participate  in  the  perfonnance  measure  using  the  travel 

time  calculations  described  above.  The  destination  block  that  is  assigned  to  an 
unavailable  RTG  by  the  neighborhood  function  is  then  reserved  until  the  next  decision 
epoch.  The  availability  window  is  defined  at  runtime  of  the  ADP. 

To  find  an  appropriately  sized  availability  window,  we  implemented  the  ADP 
look-ahead  heuristic  multiple  times,  solving  for  an  availability  window  of  zero,  then  one, 
until  an  availability  window  of  thirty  is  reached.  We  then  choose  the  best  solution  from 
this  list  of  thirty. 

3.  Container  Handling  Policy 

A  RTG  is  not  allowed  to  transit  from  a  block  until  all  work  on  available 
containers  in  its  block  or  in  its  buffer  has  been  completed.  When  handling  the  containers, 
export  containers  take  precedence  over  import  containers.  An  import  container  in  the 
buffer  has  been  discharged  from  the  container  ship  by  the  quay  crane  satisfying  a  piece  of 
the  container  sequence.  An  export  container  in  the  block  still  has  a  requirement  to  be 
loaded  onto  a  container  ship  by  the  quay  crane.  Logically,  if  there  are  containers  in  a 
block  or  buffer  to  be  handled,  the  export  containers  take  precedence  over  the  import 
containers  due  to  the  fact  export  containers  still  have  a  sequencing  constraint. 
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D.  INTEGER  LINEAR  PROGRAM  FORMULATION 


The  baseline  problem  is  formulated  as  an  integer  linear  program  (ILP).  Like 
RTGOP,  the  ILP  seeks  to  minimize  the  total  amount  of  time  until  a  given  sequence  of 
containers  have  been  moved  to  their  respective  destinations  within  the  container  terminal 
or  onboard  a  ship.  To  accomplish  the  task  of  the  container  sequence,  the  model 
sequences  RTG  movement  and  container  handling. 

The  model  is  different  from  the  DP  in  that  it  does  not  distinguish  between  a  block 
and  a  buffer  and  any  RTG  can  simultaneously  occupy  the  same  block  as  another  RTG. 
We  choose  not  to  constrain  the  RTGs  because  the  constraint  would  add  another  level  of 
complexity  to  the  model  and  the  solution  of  this  relaxed  formulation  will  adequately 
provide  a  lower  bound  for  comparison. 

The  formulation  of  the  ILP  is  as  follows,  with  sizes  of  sets  given  for  the  numerical 
example  in  Chapter  IV: 

Indices/Sets 

c  eC  Ordered  set  of  containers  to  be  handled.  ( |C|  =108) 

c  eCj  cC  Import  containers.  ( |C7 1  =54) 

c  e  CE  a  C  Export  containers.  ( | CE  |  =54) 

We  require  that  C;nC£  =  0  and  C;uC£=C  in  the  ordered  sequence  C.  We  use  the 
notation  ord(c)  to  indicate  the  ordinal  position  of  container  c  in  the  sequence  C,  and  the 
notation  c+1  to  indicate  the  next  container  in  the  sequence  C  after  container  c,  as  long  as 
ord(c)<  |C| . 


w,w'  eW  =  {1,2,..., 

\w\}  Blocks  for  RTGs. 

(1*1=36) 

g  eG  =  {1,2,...,  G|} 

RTGs. 

(|G|=6) 

je  /  =  {1,2,...,  J  } 

RTG  job  on  ordered  job  list. 

V 

II 

OJ 

o 
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Data 


conTransitTimec  w 

rtgTransitTimew  w, 

initial 

quayTime 

handleTime 

upperBound 


Time  to  move  container  c  between  the  quay  and  block  w. 

Time  to  move  a  RTG  between  block  w  and  vv'. 

Binary  data  where  1  indicates  the  initial  block  w  of  RTG  g. 

Amount  of  time  a  quay  crane  takes  to  handle  a  container. 
Amount  of  time  a  RTG  takes  to  handle  a  container. 

A  large  number  to  bound  the  nonnegative  variables. 


Nonnegative  Variables 


HSgJ,c 

CQS 


g,J,c 


RTS 


g,j,c 


CTCc 

TC 


Time  RTG  g  on  job  j  starts  handling  container  c. 

Time  quay  crane  starts  to  handle  container  c,  on  job  j  of  RTG  g. 
Time  RTG  g  on  job  j  starts  transit  to  container  c. 

Time  when  work  on  container  c  is  completed. 

Time  when  work  on  all  containers  is  completed. 


Binary  Variables 


X. 


g,j,c 


g,j,w,w 


G  . 

g,J,c,w 


1  if  container  c  is  moved  with  RTG  g  on  job  j. 

1  if  RTG  g  on  job  j  is  moved  from  block  w  to  w'. 

1  if  container  c  is  grounded  at  block  w  for  RTG  g  on  job  j. 


Objective  Function 

(1)  min  TC 
s.t 

g>r 

WE-r.j,51 

C 

c  c 
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Vg,j 

Vg,j,ord(j)<\j\ 


Vg,j,ceC 


(6)  y  p  .  ,  =  c 

v  g,J,W,W 


(7)  Y  P  .  ,  <  initial 

V  /  g,j,w,w 


Vg,  w  and  for  j  =  1 


(8) 


Vg,j,w,ord(j)<  J 


(9)Yp  .  ,=Yg  . 

v  7  g,J,w,w  Lu  gJ>C 


Vg,7,w' 


(10)  HSg  j  c  <  X g  j  cupperBound 


(11)  ’Z,ffSSJX>YdCQSSJi  c+quayTime+  5]  Gg  jc  wconTransitTimec 


Vg,_/,ceC 


VceC, 


(12)  YjHSsj,c  -  TjRTSsj,c  +  Z  pgj,w,w-rtgTransitTime» 


Vg,j 


(13)  CQSgjc  <Xg  jcupperBound 


\/g,j,ceC 


(14)^ >  X //^,/.c  +  handleTime  +  I  G’„  .  c  wcon  TransitTimec  w  \/c  eCE 


(15)  >^CQStu  +  quayTime 


Vc  e  C,ord(c)  <  C| 


(is) 


f°r  7  =  1 


(17)  RTSg  j  c  <  X g j  cupperBound 


( 1 8)  X RTSg,j+u  >  Ys(HSgj,c  +  X g  j  JiandleTime)  - 


VgJ,ceC 


(1  -  5]  j  c)upperBound 


\/g,j,ord(j)<\j\ 


(19)  CTC  =  5]  ;  c  +  handleTime 


VceC, 


(20)  CTCc  =  5]  CQSg j  c  +  quayTime 


VceC, 


(21)  TC>CTCc 


VceC 
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The  objective  function  (1)  expresses  the  total  amount  of  time  that  is  spent 
working  on  the  containers  within  a  container  tenninal.  Hence,  the  model  seeks  the 
minimum  time  it  takes  a  container  terminal  complete  all  work  on  containers  by 
sequencing  RTG  movement  and  the  handling  of  containers  in  order  to  satisfy  the 
sequence  in  which  a  quay  crane  must  handle  the  containers.  Constraint  set  (2)  ensures 
that  each  container  c  is  scheduled  to  be  handled  a  RTG  g  on  job  j.  Constraint  set  (3) 
allows  at  most  one  container  c  to  be  scheduled  for  a  RTG  g  on  job  j.  Constraint  set  (4) 
ensures  all  decisions  to  handle  containers  c  with  RTG  g  on  job  j  precede  jobs  j  that  have 
no  decision.  Constraint  set  (5)  ensures  that  the  grounding  decision  Ggj,c,w  for  container 
c  eC  is  linked  with  the  decision  XgyjyC  to  handle  the  container  with  RTG  g  on  job  j. 
Constraint  set  (6)  ensures  that  there  is  a  decision  to  move  PgjyW,w'  for  every  container  in 
the  container  sequence.  Constraint  set  (7)  assigns  the  starting  block  w  of  RTG  g  on  its 
first  job  y=l.  Constraint  set  (8)  assigns  the  destination  block  vv '  for  RTG  g  on  job  j  in 
Pgj.w.w'  as  the  starting  position  w  in  the  next  job  j+ 1 .  Constraint  set  (9)  assigns  the  ending 
block  w'  of  RTG  g  on  job  j  in  PgJ».  w  based  on  the  decision  Ggj,c,  w-  to  ground  a  container 
ceCat  vv ' .  Constraint  sets  (10),  (13)  and  (17)  bound  the  positive  variables  HSgjyC, 
CQSgjc  and  RTSgyjyC  to  be  zero  or  less  than  a  defined  upper  boundary  based  on  the  value 
of  the  binary  variable  XgyJyC.  Constraint  sets  (11)  and  (12)  coordinate  the  handle  times  for 
a  containers  c  e  C  and  a  RTG  g  on  job  j.  In  constraint  set  (11),  a  import  container’s 
c  sCj  handle  start  time  is  greater  than  its  quay  crane  start  time  plus  the  quay  handle  time 
and  the  container’s  transit  time.  Constraint  set  (12)  ensures  that  all  RTGs  g  on  job  j  have 
a  handle  start  time  greater  than  there  transit  start  time  plus  the  time  to  transit  to  the 
container  ceC.  Constraint  sets  (14)  and  (15)  set  the  lower  bound  for  the  time  that  a 
quay  crane  can  start  work  on  container  c.  More  specifically,  constraint  set  (14)  ensures 
that  a  quay  crane  does  not  start  work  on  an  export  container  ceCE  until  it  has  been 

handled  and  moved  to  the  quay  crane.  Constraint  set  (15)  ensures  that  timing  of  the 
container  sequence  is  preserved.  Constraint  set  (16)  sets  the  initial  transit  start  time  for 
RTG  g  on  job  j  to  zero.  Constraint  set  (18)  ensures  that  when  a  RTG  g  transit  start  time 
to  handle  a  container  c  eC  on  job  j+1  is  greater  than  its  handle  start  time  plus  its  handle 
time  on  job  j.  Otherwise,  the  transit  start  time  is  forced  to  zero  for  all  containers  c  not  on 
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the  RTG’s  job  list  j.  Constraint  sets  (19)  and  (20)  accumulate  the  total  time  that  it  took  to 
complete  work  on  an  import  or  export  container.  Constraint  set  (21)  sets  the  lower  bound 
for  the  objective  function  which  is  seeking  to  minimize  the  total  time  to  complete  all 
containers. 
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IV.  COMPUTATIONAL  STUDY 


A.  SCENARIO  DEVELOPMENT 

Since  RTGOP  is  solved  with  a  heuristic  algorithm  that  sacrifices  accuracy  for 
computational  speed  over  a  planning  horizon,  two  alternate  measures  of  perfonnance 
were  employed  for  comparison.  The  first  measure  is  the  baseline  ILP  developed  in 
Chapter  III  and  the  second  is  a  model  developed  by  Akel  (2007).  Data  and  container 
tenninal  layout  were  chosen  to  remain  consistent  with  Akel’s  thesis  (Figure  1).  The 
container  terminal  layout  consists  of  twelve  lanes  which  are  made  up  of  three  blocks. 
The  lanes  run  parallel  to  the  ships  berth  and  within  the  blocks  of  containers,  six  RTGs  are 
in  operation.  The  data  was  provided  by  Navis  LLC  which  consists  of  over  6,000  import 
and  export  containers  and  the  sample  of  containers  used  for  numerical  tests  was  54 
import  and  54  export  containers. 

B.  IMPLEMENTATION  AND  RUN  TIME  ANALYSIS 

With  the  scenario  developed,  we  implement  the  ADP  and  the  ILP  that  were 
developed  in  Chapter  III,  in  addition  to  the  ILP  developed  by  Akel  (2007),  for  analysis. 
All  three  models  were  run  on  a  Dell  Precision  PWS690  Intel®  Xeon™  CPU  3. 73 GHz 
processor,  with  3.00GB  of  RAM. 

1.  Approximate  Dynamic  Program 

The  ADP  was  implemented  and  run  on  Java  Platform  Version  1.6.0  with  Eclipse: 
Version  3. 3. 1.1  as  the  integrated  development  environment. 

Since  we  made  the  choice  to  focus  on  RTG  travel  within  the  yard  of  a  container 
tenninal,  we  will  first  look  at  the  RTGOP  heuristic  output  that  defines  the  movement  of 
RTGs.  When  looking  at  the  movement  of  the  RTGs  compared  to  the  timeline,  we  can  see 
how  an  RTG  moves  around  the  tenninal  from  block  to  block  and  lane  to  lane.  Initially, 
the  heuristic  does  well  at  keeping  RTGs  within  a  single  lane  or  moves  RTGs  to  nearby 

lanes.  However,  just  after  the  75  minute  mark,  all  export  containers  have  been  moved  out 
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of  their  blocks  and  are  waiting  to  be  loaded  onto  the  container  ship  by  the  quay  crane. 
The  result  of  the  event  is  that  the  RTGs  are  forced  to  move  to  different  locations  in  the 
yard  to  handle  the  import  containers.  After  the  quay  crane  stops  work,  the  movement  of 
RTGs  in  the  yard  settles  and  RTGs  finish  work  on  the  remaining  import  containers  in  the 
yard. 


RTG  Position  i 

n  Yard  (Look-Ahead  16  min) 

All  Export  containers  moved 
out  of  blocks  by  RTGs. 

Quay  crane  has 
completed  work 

All  work  on  containers 
has  been  completed. 
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Figure  8.  RTG  Movement  in  Time  (16  Minute  Look-ahead):  The  movement  of 
the  RTGs  that  resulted  from  planning  moves  for  RTGs  that  had  less  than  16  minutes  of 
work. 


Comparing  a  solution  with  no  look-ahead  to  a  solution  with  a  16  minute  look 
ahead,  we  see  that  the  RTGs  are  making  larger  moves  more  often  with  no  look-ahead. 
The  time  that  the  RTGs  finish  work  on  the  export  containers  is  about  the  same,  however, 
all  the  RTGs  converge  to  lane  three  at  about  the  120  minutes  in  the  no  look-ahead 
solution.  This  convergence  of  the  RTGs  combined  with  the  longer  moves  slows  the  quay 
crane  such  that  it  finishes  later  (Figure  9).  The  solutions  in  both  Figure  8  and  9 
demonstrate  the  importance  of  a  look-ahead  heuristic. 
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RTG  Position  in  Yard  (No  Look-Ahead) 


All  Export  containers  moved 
out  of  blocks  by  RTGs. 
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Quay  crane  has 
completed  work 


All  work  on  containers 
has  been  completed. 
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Figure  9.  RTG  Movement  in  Time  (No  Look-Ahead):  This  Chart  represents  no 
future  planning  of  RTGs. 


Within  the  ADP,  the  time  to  solve  is  largely  dependant  on  the  size  of  the 
approximate  optimal  value  ( t )  and  yields  a  linear  run  time  of  0(7) .  With  each  unit  of 
increase  in  the  optimal  value  the  number  of  computations  required  in  the  ADP  is 
increased  by  approximately  47,000.  The  majority  of  the  computations  per  iteration  is 
attributed  to  the  RTG  neighborhood  policy,  making  the  neighborhood  function  the  most 
computationally  expensive  piece  of  the  ADP  algorithm. 

In  the  developed  scenario  for  numerical  trials,  the  number  of  RTGs  remains  at  six. 
If  the  number  of  RTGs  used  in  the  scenario  is  not  constant,  then  the  run  time  becomes 
dependant  on  the  number  of  RTGs.  The  dependency  between  run  time  and  the  number 
of  RTGs  is  found  in  the  neighborhood  policy.  The  approximate  number  of  computations 
in  the  neighborhood  policy  is  a  constant  46,700.  When  the  number  of  RTGs  is  allowed  to 
vary,  the  run  time  of  the  combinatorial  search  for  a  policy’s  RTG  travel  times  becomes 


29 


0(n")  where  n  is  the  number  of  RTGs.  Varying  RTGs  within  the  scenario  was  not 
considered  and  will  have  dramatic  effects  on  run  time. 


ADP  Run  Times 


Figure  10.  RTGOP  ADP  Run  Times:  For  the  scenario,  runtimes  of  the 
neighborhood  policies  are  compared  across  different  planning  horizons. 

The  run  time  of  the  ADP  is  extremely  important  when  looking  at  the  objective  of 
real-time  dispatching  of  RTGs.  As  in  the  fonnulation  of  RTGOP,  the  frequency  of  the 
actual  decision  epoch  within  a  terminal  is  determined  by  the  quay  crane  push  rate. 
RTGOP  decisions  have  to  be  determined  in  the  time  between  a  terminal’s  decision 
epochs.  When  comparing  the  neighborhood  policies,  all  policies  in  the  neighborhood 
function  combined  with  a  sizable  planning  horizon  would  meet  the  requirement  of  even 
the  most  aggressive  quay  crane  push  of  30  seconds  (Figure  10).  As  we  can  see  from 
Figure  10,  the  minimum  maximum  path  with  a  penalty  is  just  under  a  30  second  solve 
time  for  an  50  container  planning  horizon  and  the  other  two  destination  selection  criteria 
solve  for  a  100  container  planning  horizon  under  30  seconds. 
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ADP  Approximately  Optimal  Values 


Figure  11.  RTGOP  ADP  Approximate  Optimal  Values:  The  approximately 
optimal  values  for  the  different  neighborhood  policies  are  computed  for  the  different 
planning  horizons  for  comparison. 

Since  the  run  times  of  the  different  neighborhood  policies  are  fairly  comparable, 
the  optimal  values  are  used  to  distinguish  their  performance.  We  use  regression  analysis 
to  identify  the  neighborhood  policy  that  on  average  yielded  the  lowest  optimal  values. 
The  policy  that  preformed  the  best  for  the  given  scenario  and  data  was  the  minimum 
maximum  shortest  path  (Figure  11).  Later  in  this  chapter,  RTGOP  with  the  minimum 
maximum  shortest  paths  is  used  for  comparison  to  the  baseline  models. 

2.  ILP  Analysis 

Both  integer  linear  program  models  were  implemented  in  GAMS  and  found 
optimal  or  near-optimal  solutions  using  the  CPLEX  solver  version  10.0.1(ILOG,  2007). 

a.  Baseline  ILP  from  Chapter  III 

Since  the  computational  complexity  of  the  baseline  ILP  for  the  movement 

of  RTGs  within  a  container  terminal  is  Nondeterministic  Polynomial-time  hard,  the 
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number  of  containers  and  RTG  moves  that  could  be  practically  calculated  for  comparison 
is  small.  The  number  of  exact  solutions  we  were  able  produce  on  our  timeline  was  nine. 
The  solve  time  for  a  nine  container  planning  horizon  was  over  19  hours  (Figure  12).  For 
a  ten  container  planning  horizon,  the  CPLEX  algorithm  ran  for  48  hours  resulting  in  a 
solution  with  a  15.79%  relative  gap.  Based  on  the  fact  that  run  times  were  getting 
progressively  worse,  it  was  not  practical  to  proceed  further  in  this  direction. 


ILP  Run  Times 


Container  Planning  Horizon 


Figure  12.  Baseline  ILP  Run  Times:  Run  times  to  find  an  exact  solution  of  the 
different  planning  horizons  are  compared.  The  graph  shows  that  run  time  gets 
progressively  worse  as  the  planning  horizon  is  increased. 

The  small  set  of  optimal  values  in  the  solution  of  the  ILP  provided  us  with 

some  grounds  for  comparison  to  the  RTGOP  ADP  solutions  but  the  quantity  of  solutions 

is  insufficient.  The  run  time  behavior  may  prevent  us  from  finding  solutions  for  the 

entire  108  container  horizon,  but  solution  to  the  relaxation  of  the  ILP  model  to  a  linear 

program  (LP)  provided  us  with  an  optimistic  lower  bound  for  comparison  to  the  RTGOP 

ADP.  We  solved  for  58  planning  horizons  which  had  a  combined  total  solve  time  of  over 

150  hours  (Figure  13).  The  worst  case  run  time  was  twenty  hours  for  a  56  container 
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planning  horizon.  When  comparing  the  exact  solutions  of  the  ILP  to  the  LP  relaxation, 
the  optimal  values  were  exactly  equal.  The  absolute  best  case  would  be  if  the  exact 
solution  continues  to  equal  the  LP  relaxation  for  future  values. 


ILP  Relaxation 


Container  Planning  Horizon 

Figure  13.  Optimal  Values  of  the  Baseline  ILP  Relaxation:  The  optimal  values 
computed  for  the  different  planning  horizons  provide  an  absolute  lower  bound  to  our 
problem. 


b.  Previously  Developed  ILP  (Akel,  2007) 

In  the  implementation  of  the  Akel’s  model,  five  different  planning 
horizons  were  chosen  for  points  of  comparison  to  the  ADP.  The  Cascade  Method  was 
the  optimization  based  heuristic  method  used  to  solve  Akel’s  formulation  (Baker  et  al. 
1998).  In  the  cascade,  the  problem  was  divided  into  equal  sub  problems.  The  optimal 
solutions  from  the  first  sub  problem  are  fixed  and  the  subsequent  sub  problems  are  re¬ 
optimized  including  the  fixed  solutions  to  the  prior  sub  problems.  The  process  is 
continued  until  all  the  sub  problems  are  solved.  Each  sub  problem  has  an  equally 
complex  portion  of  the  original  problem,  giving  solution  technique  a  linear  run  time. 
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The  heuristic  solution  methodology  allowed  the  run  time  to  be 
significantly  faster  than  our  baseline  ILP,  finding  a  solution  for  a  60  container  planning 
horizon  within  three  minutes  (Figure  14).  However,  when  considering  the  real-time 
dispatching  of  RTGs,  the  model  will  not  be  able  to  keep  up  pace  with  a  slow  quay  crane 
push  rate  of  90  seconds.  The  run  time  of  CPLEX  at  a  planning  horizon  of  10  containers 
is  100  seconds.  The  output  of  Akel’s  model  provides  an  additional  benchmark  for  the 
RGTOP  ADP  optimal  values. 


Figure  14.  Prior  Model  Run  Time  (After  Akel,  2007):  Prior  model  runtime 
increases  linearly  as  the  planning  horizon  increases 

C.  COMPARATIVE  ANALYSIS  OF  OPTIMAL  VALUES 

In  the  comparison  of  the  optimal  values,  the  three  models  are  using  the  same 
container  terminal  layout,  number  of  RTGs,  and  sequenced  container  data.  Where  the 
models  are  different  are  the  constraints  that  are  placed  on  container  terminal  operations. 

The  baseline  model,  the  ILP  fonnulated  in  Chapter  III,  is  the  least  restrictive 
implementation  of  container  terminal  operations.  The  formulation  simply  establishes  a 
grounding  policy  and  sequences  the  RTG  movement  and  handling  of  containers  to  satisfy 
the  given  container  sequence.  The  formulation  of  the  DP  accomplishes  the  same  task 
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with  an  addition  of  one  restriction.  RTGs  are  not  allowed  to  occupy  the  same  block.  In 
Akel’s  model  (2007),  the  fonnulation  is  much  more  restrictive.  The  RTGs  were 
restricted  in  the  number  of  moves  that  they  could  make  in  handling  the  containers,  RTGs 
were  assigned  to  a  specific  set  of  work  blocks,  all  of  the  RTGs  perfonned  an  equal 
measure  of  work  and  RTGs  are  not  allowed  to  handle  the  next  container  in  sequence  if 
they  are  currently  working  on  a  container. 


Figure  15.  Comparison  of  Objective  Values:  The  graph  compares  the  optimal 
values  and  trends  of  the  three  models  across  planning  horizons. 

Being  the  most  restrictive,  Akel’s  model  provides  an  upper  bound  on  the  optimal 
values  for  the  ADP.  On  the  contrary,  LP  relaxation  provides  an  absolute  lower  bound  on 
the  optimal  values  for  the  RTGOP  ADP  (Figure  15).  To  our  satisfaction,  the  values 
produced  by  the  ADP  fall  in  between  the  bounds  set  by  the  other  models. 

The  gap  between  the  ADP  and  the  ILP  solutions  appears  to  be  consistent,  at 
approximately  60  minutes.  For  10  hours  of  work,  if  the  gap  remains  consistent,  then  our 
implementation  provides  a  very  effective  and  efficient  solution  to  this  complex  problem. 
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Finally,  we  can  surmise  that  the  real  value  of  the  optimal  solution  is  than  that  provided  by 
the  ILP,  since  the  ILP  is  a  relaxation  of  our  problem  and  allows  multiple  RTGs  to  occupy 
the  same  block  at  the  same  time. 
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V.  CONCLUSION  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

The  thesis  is  tackling  the  problem  of  real-time  dispatching  of  assets  in  a  container 
tenninal  as  well  as  the  grounding  policy  for  import  containers.  As  demonstrated  in 
Chapter  IV,  the  problem  of  optimally  sequencing  RTG  movement  and  its  handling  of 
containers  to  meet  the  constraint  of  a  container  sequence  is  computationally  expensive. 
Even  if  the  problem  could  be  solved  optimally  in  a  timely  manner,  an  unexpected  delay 
or  machine  failure  can  change  the  optimal  sequence  which  leads  to  suboptimal 
performance.  The  problem  would  need  to  be  resolved. 

In  this  thesis,  we  develop  a  detenninistic  dynamic  program  for  the  real-time 
dispatching  problem  that  can  quickly  be  solved  using  approximate  dynamic  programming 
(ADP).  Additionally,  the  ADP  heuristic  can  be  quickly  resolved  in  the  presence  of  data 
changes.  Based  on  runtime  analysis  alone,  it  is  clear  that  the  APD  based  heuristic  is 
suited  for  the  task  of  real-time  dispatching  of  RTGs  in  conjunction  with  a  grounding 
policy.  Not  only  do  the  rules  within  the  policy  of  the  ADP  allow  the  program  to 
converge  to  a  solution  set  quickly,  but  the  algorithm  combined  with  a  sizable  container 
planning  horizon  has  the  ability  to  keep  up  with  an  aggressive  decision  making  process 
where  the  quay  crane  push  rate  is  30  seconds. 

As  shown  at  the  end  of  Chapter  IV,  trends  of  the  approximately  optimal  values  of 
the  ADP  based  heuristic  are  closely  related  to  the  optimal  values  found  in  by  the  linear 
programming  relaxation  of  a  bench-mark  ILP.  This  is  important  because  it  shows  that  as 
the  planning  horizon  increases,  the  optimal  values  from  the  ADP  heuristic  and  the  ILP 
head  in  the  same  direction  with  very  little  divergence.  Additionally,  we  surmise  that  the 
growth  of  the  gap  remain  relatively  consistent  as  the  container  horizon  increases  which 
gives  a  very  optimistic  outlook  for  large  container  sequences. 
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B. 


FUTURE  WORK 


In  this  thesis,  we  develop  an  ADP  based  heuristic  algorithm  that  has  the  ability  to 
produce  real-time  solutions.  Additionally,  we  combine  the  grounding  policy  with  yard 
asset  optimization  when  these  two  have  traditionally  been  solved  separately.  With  the 
accomplishments  made  in  this  thesis,  development  of  multiple  scenarios  is  identified 
further  study. 

During  the  numerical  trial  of  the  ADP  heuristic,  a  single  scenario  was  developed 
for  analyzing  feasibility  of  real-time  dispatching  and  initial  implementations  of  terminal 
policies.  Several  different  scenarios  with  unplanned  delays  and  mechanical  failures  are 
needed  for  more  advanced  numerical  trials.  The  multiple  scenarios  will  aid  in  improving 
the  perfonnance  of  the  heuristic  in  different  terminal  layouts,  changing  environments  and 
large  container  sequences. 
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